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' Abstract 

A version of the Dynamical Systems Method (DSM) for solving ill-conditioned linear 
• algebraic systems is studied in this paper. An a priori and a posteriori stopping 

rules are justified. An iterative scheme is constructed for solving ill-conditioned linear 
■ algebraic systems. 
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1 Introduction 

O : 

We want to solve stably the equation 

rn ■ 

O ■ Au = f, (1) 

OO . 

where A is a linear bounded operator in a real Hilbert space H. We assume that (HJ has 
a solution, possibly nonunique, and denote by y the unique minimal-norm solution to (pQ), 
y _L M := M{A) := {u : Au = 0}, Ay = f. We assume that the range of A, R(A), is not 
closed, so problem ([1]) is ill-posed. Let fg, \\f — fs\\ < $, be the noisy data. We want to 
construct a stable approximation of y, given {5, f$, A}. There are many methods for doing 
this, see, e.g., [3|— [B], [7j, [13], [T3], to mention some (of the many) books, where variational 
regularization, quasisolutions, quasiinversion, and iterative regularization are studied, and 
[7]- [T2], where the Dynamical Systems Method (DSM) is studied systematically (see also 
[I]) P3]> P3L an d references therein for related results). The basic new results of this paper 
are: 1) a new version of the DSM for solving equation (pQ) is justified; 2) a stable method 
for solving equation (pQ) with noisy data by the DSM is given; a priori and a posteriori 
stopping rules are proposed and justified; 3) an iterative method for solving linear ill- 
conditioned algebraic systems, based on the proposed version of DSM, is formulated; its 
convergence is proved; 4) numerical results are given; these results show that the proposed 
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method yields a good alternative to some of the standard methods (e.g., to variational 
regularization, Landweber iterations, and some other methods). 

The DSM version we study in this paper consists of solving the Cauchy problem 

u(t) = -P{Au{t) - /), u(0)=u o , u ±M, u:=—, (2) 

and proving the existence of the limit lim^ooU^) = u(oo), and the relation u(oo) = y, 
i.e., 

lim \\u(t) - y\\ = 0. (3) 

Here P is a bounded operator such that T := PA > is selfadjoint, Af(T) = M(A). 

For any linear (not necessarily bounded) operator A there exists a bounded operator 
P such that T = PA > 0. For example, if A = U\A\ is the polar decomposition of 
A, then \A\ := (A*A)a is a selfadjoint operator, T := \A\ > 0, U is a partial isometry, 
\\U\\ = 1, and if P := U* , then ||P|| = 1 and PA = T. Another choice of P, namely, 
P = (A* A + al^A*, a = const > 0, is used in Section 

If the noisy data fs are given, \\fs — f\\ < S, then we solve the problem 

u 5 {t) = -P{Au s {t) - f 5 ), u s (0) = no, (4) 

and prove that, for a suitable stopping time t$, and u$ := u$(ts), one has 

lim \\us — y\\ = 0. (5) 
<5^0 

An a priori and an a posteriori methods for choosing tg are given. 

In Section 2 these results are formulated and recipes for choosing tg are proposed. In 
Section [3] a numerical example is presented. 



2 Formulation and results 

Suppose A : H — > is a linear bounded operator in a real Hilbert space H. Assume that 
equation (JTJ) has a solution not necessarily unique. Denote by y the unique minimal-norm 
solution i.e., y _L J\f := M(A). Consider the DSM where uq -L is arbitrary. Denote 

T := PA, Q := AP. (6) 

The unique solution to (J2j) is 

u (t) = e - tT u + e~' T I e sT dsPf. (7) 

Let us first show that any ill-posed linear equation ([1]) with exact data can be solved by 
the DSM. 
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2.1 Exact data 

The following result is known (see [7]) but a short proof is included for completeness. 

Theorem 1 Suppose uq _L N and T* = T > 0. Then problem ([2]) has a unique solution 
defined on [0, oo), and u(oo) = y, where u(oo) = lim^oo u(t) . 

Proof. Denote w := u(t) — y, wo := w(0) = uq — y. Note that wo _L M. One has 

w = -Tw, T := PA, w(0) =u -y. (8) 

The unique solution to (JSJ) is w = e~ tT WQ. Thus, 

r\\n 

\\ w \\ 2 = / e - 2tx d(E x w ,w ). 
Jo 

where (u, v) is the inner product in H, and E\ is the resolution of the identity of T. Thus, 

|Moo)|| 2 = lim I " e- 2tx d{E x w ,w ) = \\PjfWof = 0, 
where Pj\f = Eq — E_o is the orthogonal projector onto M. Theorem Q] is proved. □ 

2.2 Noisy data f s 

Let us solve stably equation ([1]) assuming that / is not known, but fg, the noisy data, are 
known, where \\fs — f\\ < S. Consider the following DSM 

ii s = -P(Au s - f s ), u s (0) = uq. (9) 

Denote 

w s :=u s -y, T := PA, w s (Q) = w := u - y £ J\f ± . 
Let us prove the following result: 

Theorem 2 IfT = T*>0, lim^o^ = °°i lim^ >ot<5<5 = 0, and wq £ M^, then 

lim 11^(^)11 = 0. 

Proof. One has 

w s = -Tw s + ( s , Cs = P(fs~f), WCsW < \\P\\S. (10) 
The unique solution of equation (fTUj) is 

w 5 (t) = e- tT w s (0) + f e'^ T Csds. 
Jo 
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Let us show that lim^o Hto^i^) || =0. One has 



lim \\w s (t)\\ < lim \\e~ tT w s (0)\\ + lim 

t— >oo t— >oo t— >oo 



-(t-«)T 



(11) 



Let .Ea be the resolution of identity corresponding to T. One uses the spectral theorem 
and gets: 



fe-^ T dsQ= f f m dE x Cse- (t - s »ds 
Jo Jo Jo 



n „u_i 

A Jo 



n 1 _ e -tx 



dE x ( 5 . 



Note that 



1 - e~ tx 

< < t, VA > 0, t > 0, 

A 



since 1 — x < e x for x > 0. From (1121) and (fT3l), one obtains 



2 ,HT[| i_ e -*A 



d{E x ( 5 ,( S ) 



<t 2 [ d(E x Cs,Cs) 
Jo 

= t 2 \\Cs\\ 2 . 

Since ||C<5|| < ||-P||<5, from (fTT|) and (fH|) . one gets 



lim K(t*)|| < lim ( \\e- tsT w s (0)\\ + t s 5\\P\ 



0. 



Here we have used the relation: 



hm\\e- tlT w s (0)\\ = \\Pmwo\\=0, 

and the last equality holds because wq G Af- 1 . Theorem [2] is proved. □ 

From Theorem [21 it follows that the relation 

C 

ts = jf, 7 = const, 76 (0,1) 

where C > is a constant, can be used as an a priori stopping rule, i.e., for such ts one 
has 

lim \\us(ts) - y\\ = 0. (15) 



(12) 



(13) 



(14) 
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2.3 Discrepancy principle 

In this section we assume that A is a linear finite-rank operator. Thus, it is a linear 
bounded operator. Let us consider equation ([T]) with noisy data /<$, and a DSM of the 
form 

u s = -PAu 5 + Pfs, u*(0) = n . (16) 

for solving this equation. Equation (|16|) has been used in Section |2T2"1 Recall that y denotes 
the minimal-norm solution of equation ([I]) . Example of a choice of P is given in Section [3j 

Theorem 3 Let T := PA, Q := AP. Assume that \\Au - f s \\ > CS, Q = Q* > 0, 
T* = T > 0, T is a finite-rank operator. Let M{T) =: M. Note that M{T) = M{A). The 
solution ts to the equation 

h{t) := \\Au s (t) - f s \\ = CS, C= const, C £ (1,2), (17) 

does exist, is unique, and 

lim\\us{ts)-y\\=0, (18) 
where y is the unique minimal-norm solution to ([T|). 
Proof. Denote 

v s (t) := Aus(t) - f s , w(t) := u(t) - y, w :=u -y. 



One has 



j t \\v 5 (t)\\ 2 = 2(Au 5 (t),Au 5 (t) - f 5 ) 



(19) 



= 2(A[-P(Aus(t) - f s )],Aus(t) - fs) 

= -2(AP(Au s - fs),Au s - fs) < 0. 

where the last inequality holds because AP = Q > 0. Thus, ||v5(t)|| is a nonincreasing 
function. 

Let us prove that equation fjlTf) has a solution for C G (1,2). One has the following 
commutation formulas: 

e -sT p = p e sQ^ A( ,-sT = e sQ A 
Using these formulas and the representation 



u s (t) 



- tT u + fe~^ T Pfsds, 
Jo 



one gets: 



v 5 {t) = Au s (t) - f 5 

= Ae- tT u + A f e~^ T Pf s ds - f s 
Jo 

= e~ tQ Au + e- tQ [ e sQ dsQf s - fs 
Jo 

= e -tQ A (u -y) + e~ tQ f + e- tQ (e tQ - L)f s - fs 
= e- tQ Aw - e~ tQ fs + e~ tQ f = e~ tQ Au - e~ tQ f 5 . 



(20) 
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Note that 



lim e tQ Awo = lim Ae tT wo = APj^wq = 0. 

t^oo t^oo 



Here the continuity of A and the following relation 

I|T|| 

^ im e~ tT wn = lim / 
were used. Therefore, 



r\\T\\ 

lim e~ tT wo = lim / e st dE s wo = (Eq — E-q)wq = Pj^/wq, 



lim \\v s (t)\\ = lim ||e- t( 3(/ - f s )\\ < \\f - f s \\ < 5, (21) 

t^oo t— >oo 

where ||e -t< *|| < 1 because Q > 0. The function h(t) is continuous on [0, oo), h(0) = 
|| Aixo — fs\\ > C<5, /i(oo) < 5. Thus, equation (fT7|) must have a solution i^. 

Let us prove the uniqueness of tg. If tg is non- unique, then without loss of generality 
we can assume that there exists t\ > tg such that ||Ait<5(ii) — fg\\ = C6. Since ||^(i)|| is 
nonincreasing and H^^)!! = ||«,y(ii)||, one has 

||«i(t)|| = \\v s (t s )\\, Vt G [tg,ti]. 

Thus, 

j t h s (t)\\ 2 = o, vte(t s ,h). (22) 

Using (fT9|) and ([22]) one obtains 

||VAP(Aui(i) - /,5)|| 2 = (AP(Aug(t) - fg),Au s (t) - f s ) = 0, Vt G [t,,ti], 

where \J AP = Q? > is well defined since Q = Q* > 0. This implies (^(Au^ — fs) = 0. 
Thus 

Q(Au*(t) - f s ) = 0, Vte[ts,h}- (23) 

From (|20p one gets: 

«i(t) = Att,(t) - A = e- tQ Au - e~ tQ f 8 . (24) 

bmce 

Qe'tQ = e -tQQ and e"* is an isomorphism, equalities (123f) and (|24|) imply 

Q(An - fs) = 0. 

This and (|24p imply 

AP(Au a (i) - /,) = e-* Q (QAno - Qf s ) = 0, t > 0. 
This and (|19j) imply 

^IM| 2 = 0, t>0. (25) 
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Consequently, 

C5 < \\Au e (0) - M| = |k(0)|| = \\v s (ts)\\ = \\Au s (t s ) - f 5 \\ = CS. 

This is a contradiction which proves the uniqueness of t$- 
Let us prove (|18p . First, we have the following estimate: 



\\Au(t s ) - /|| < \\Au(t s ) - Aus(t s )\\ + \\Aus(t s ) - f s \\ + \\fs - f\\ 



< 



tsQ / e sQ 



e^Qds 



\\fs-f\\+C5 + 5, 



(26) 



where u(t) solves ([2]) and us(t) solves Q. One uses the inequality: 



-t s Q I 5 e sQQds\\ = ||j - e - tsQ \\ < 2, 

JO 



and concludes from (1261). that 



Secondly, we claim that 



lim||A«(ti)-/||=0. 

o— >U 



lim ts = oo. 

8^0 



(27) 



Assume the contrary. Then there exist to > and a sequence (ts n )^ = i, t$ n < to, such that 

lim \\Au(t 5n )-f\\=0. (28) 

n— >oo 

Analogously to (fT9j) . one proves that 

d „ ,,9 

where v(t) := Au(t) — f. Thus, ||v(i)|| is nonincreasing. This and (|28p imply the relation 
||^o)|| = \\Au(t )-f\\ =0. Thus, 

= v(t o ) = e- toQ A(u -y). 

This implies A(u - y) = e* oQ e _ * oQ A(ti - y) = 0, so u - y G AA. Since u - y e Af- 1 , it 
follows that uq = y. This is a contradiction because 



Thus, 



CS < \\Au - fs\\ = ||/ - M| < 8, 1<C<2. 



lim tx = oo. 

5^0 



(29) 



Let us continue the proof of (fT%|) . From (|2U|) and the relation HAu^a) — Ml = C<5, 
one has 



C5t s = \\t s e-^Aw - t s e- tsQ (f s - f)\\ 

< \\tse- tsQ Aw \\ + \\tse- tsQ (fs- f)\\ 

< \\t s e~ tsQ Aw \\ +t s 6. 



(30) 
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We claim that 

lim t 6 e~ tsQ Aw = lim t s Ae- tsT w = 0. (31) 
<5->0 <5^0 

Note that (|31j) holds if T > has finite rank, and wq G AA -1 -. It also holds if T > is 
compact and the Fourier coefficients woj := (wo,(f>j), T<pj = \j<j)j, decay sufficiently fast. 
In this case 

\\Ae- tT w \\ 2 < \\Th- tT w \\ 2 = ^2\ j e~ 2X i t \w 0j \ 2 := S = o(^), t -» oo, 

3=1 ' f 

provided that Y^jLi \ w 0j\^J 2 < °°- Indeed S 1 = X^a < 1 + Sa > 1 '■= S± + £>2- One has 
^i<^ E 2 ^ = °^)' S a <oe- 2 **= (i) I t-oo, 

Aj<t~§ 5 

where c > is a constant. 

From (|3~T1) and (j30j) . one gets 



< lim(C - l)St 6 < lim \\t s e~ tsQ Aw \\ = 0. 
5^0 5^0 



Thus, 



lim 5t s = (32) 
Now (|18p follows from (|29p . (|32p and Theorem [2j Theorem 3 is proved. □ 

2.4 An iterative scheme 

Let us solve stably equation (pQ) assuming that / is not known, but f$, the noisy data, are 
known, where \\fs — f\\ < 5. Consider the following dicrete version of the DSM: 

Un+l,6 = Un,6-hP(Au n ,8 — fg), USfi = U - (33) 

Let us denote u n := u nj s when 5^0, and set 

w n :=u n -y, T := PA, Wo := Uo — y £ M ± . 
Let n = ns be the stopping rule for iterations (j33|) . Let us prove the following result: 



Theorem 4 Assume that T = T* > 0, h\\T\\ < 2, lim,5_>o n <5^ = oo, lim^o n§hS = 0, 
and wo 6 M ± . Then 

lim llu^JI = lim \\u„. — y\\ = 0. (34) 
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Proof. One has 

w n+ i = uu n - hTw n + h( s , (s = P(fs-f), \\(s\\ < \\P\\5, w = u -y. (35) 
The unique solution of equation ([35]) is 



u n+1 = (I — hT) n+1 w + h ]T(I - hTYQ- 
Let us show that lining ||if n<5 1| = 0. One has 



i=0 



\w n \\ < \\(I-hT) n w \\ + 



n-l 



i=0 



(36) 



Let E\ be the resolution of identity corresponding to T. One uses the spectral theorem 
and gets: 

n— 1 n— 1 /•IITII 



h^2{I-hTy = hJ2 (1-hXydEx 
i=0 i=0 ^° 



/i 



Note that 



< 



i - (i - \hy 

1 - (1 - /iA) 
1 - (1 - /iA) n 



1 - (1 - A/i) ? 



A 



A 



< hn, VA > 0, t > 0, 



(37) 



(38) 



since 1 — (1 — a) n < an for all a £ [0, 2]. From (j37H and (I38p . one obtains 



n-l 2 /.|| 

= / 

~n JO 



l-(l-\h) n , 2 



A 



ITll 



d(E x ( s ,( s ) 



(39) 



< (/in) 2 

= (nh) 2 \\( 5 \\ 2 . 
Since < \\P\\S, from (i36l) and (f39l) . one gets 



lim ||w n J < Jhn I ||(J - fcT) n «Ti;i(0)[| + /tn 5 5||P|M = 0. 

Here we have used the relation: 

lim ||(J - hT) n * w s {0)\\ = \\Pj^w \\ = 0, 

and the last equality holds because wq G Af ± . Theorem H] is proved. 

From Theorem HI it follows that the relation 

C 

n& = j^, 7 = const, 7 G (0, 1) 

where C > is a constant, can be used as an a priori stopping rule, i.e., for such ns one 
has 



□ 



lim \\u nx — y\\ = 0. 



(40) 
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2.5 An iterative scheme with a stopping rule based on a discrepancy 
principle 

In this section we assume that A is a linear finite-rank operator. Thus, it is a linear 
bounded operator. Let us consider equation (pQ) with noisy data fg, and a DSM of the 
form 

u n+1 = u n - hP(Au n - f 5 ), u = u , (41) 

for solving this equation. Equation (|4ip has been used in Section [2~4l Recall that y denotes 
the minimal-norm solution of equation ([1]) . Example of a choice of P is given in Section [3j 
Note that M := Af(T) = N{A). 

Theorem 5 Let T := PA, Q := AP. Assume that \\Au - f s \\ > CS, Q = Q* > 0, 
T* = T > 0, h\\T\\ < 2, h\\Q\\ < 2, and T is a finite-rank operator. Then there exists a 
unique n$ such that 

\\Au ns -f s \\<C8< \\Au ns ^ -fe\\, C= const, C e (1, 2). (42) 

For this rig one has: 

lim \\u ng - y\\ = 0. (43) 

Proof. Denote 

v n := Au n - fs, w n := u n - y, w := u - y. 

From (14ip . one gets 

v n+ i = Au n+1 - fs = Au n - fs- hAP(Au n - f s ) = v n - hQv n . 

This implies 

lbn+l|| 2 - \\v n \\ 2 = (v n +l ~ V n ,V n+ \ + V n ) 

= (—hQv n , v n - hQv n + V n ) (44) 
= -(v n ,hQ{2-hQ)v n ) <0 

where the last inequality holds because AP = Q > and \\hQ\\ < 2. Thus, (H^nlD^Li ^ s a 
nonincr easing sequence. 

Let us prove that equation (|4*2|) has a solution for C £ (1,2). One has the following 
commutation formulas: 

(I - hT) n P = P(I - hQ) n , A(I - hT) n = {I- hQ) n A. 

Using these formulas, the representation 

n-l 

Un = (J - hT) n Uo + hJ2(i - hrypfs, 

8=0 
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and the identity (J - B) Y™=o B l = I — B n , with B = I - hQ, I - B = hQ, one gets: 
v n = Au n - f s 

n-l 

= A(I - hT) n u + AhY J (i- hrypfs - fs 



i=0 



n-l 



(/ - hQ) n Au + ]T(I - hQfhQf 5 - f s 



(45) 



i=0 



= (/ - hQ) n Au -(I -(I- hQ) n )fs - fs 
= (I- hQ) n (Au -/) + (/- hQ) n (f - fs) 
= (I- hQ) n Aw + (I - hQ) n (f - f 5 ). 

If V = V* > is an operator with ||V|| < 2, then ||7 — V\\ = sup 0<s<2 |1 — s| < 1. 
Note that 

lim (/ - hQ) n Aw = lim Ail - hT) n w = AP N w = 0, 

n— +oo n— >oo 

where Pj\f is the orthoprojection onto the null-space ftf of the operator T, and the conti- 
nuity of A and the following relation 

lim (I - /iT) n w;o = lim /" "(1 - s/i) n d£ s t«o = (£ - £-o)i«o = ^0, < sh < 2, 

n— >oo n— >oo Jq 

were used. Therefore, 



lim \\vs 

71— >00 



lim \\(I-hQT{f-f 5 )\\ < ||/ -M| <S, 



(46) 



where \\I — hQ\\ < 1 because Q > and ||/iQ|| < 2. The sequence {H^nll}^! is nonin 
creasing with ||«q|| > C5 and linin^oo ||w n || < 5. Thus, there exists ns > such that ([321 
holds. 

Let us prove (f4~3|) . Let u n) o be the sequence defined by the relations: 

Un+1,0 = u n,0 ~ hP{Au nfi - /), M ,0 = UQ- 

First, we have the following estimate: 

ill + 11/* -/II 

ng—l 

< 



\Au nSi0 - /|| < \\Au ns - Au n6) o\\ + \\Au ns - 

ng—l 

\\fs-f\\+C5 + 5. 

i=0 



J2(I- hQfhQ 

i=0 

Since < hQ < 2, one has ||J — hQ\\ < 1. This implies the following inequality: 

ns— 1 

-hQfhQ =\\I - (I -hQ) ns \\<2, 

i=0 



(47) 
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and concludes from (|47p . that 



Secondly, we claim that 



lim \\Au n5 , - /|| = 0. (48) 

o— >0 



lim hnx = oo. 

Assume the contrary. Then there exist no > and a sequence (ns n )'^' =1 , n^ n < n , such 
that 

lim ||Au n4i0 - /|| = 0. (49) 



Analogously to (I44j) . one proves that 

IK,o|| < ||«n-l,o||> 

where t> n ,o = Au n fi — /. Thus, the sequence ||i>n,o|| is nonincreasing. This and (j4"9j) imply 
the relation ||v no ,o|| = 11^4^0,0 — /II = 0- Thus, 

= v nOtO = (I-hQ) n °A(u o -y). 

This implies A{u - y) = (I - hQ)- n °(I - hQ) n °A(u - y) = 0, so u - y € M. Since, by 
the assumption, uq — y £ M- 1 , it follows that no = y. This is a contradiction because 

CS < \\Au - fs\\ = \\f - fs\\ < S, 1<C<2. 

Thus, 

lim hnx = oo. (50) 
Let us continue the proof of ([4*3]) , From (f4"5"|) and ||^4n„, 4 — = C(5, one has 

C5n s h = \\n s h(I - hQ) ns Aw - n s h{I - hQ) ns (f s - f)\\ 

< \\n s h(I - hQ) ns Aw \\ + \\n s h(I - hQ) n *(f s - f)\\ (51) 

< \\n s h(I - hQ) ns Aw \\ + n s h5. 

We claim that if w £ M^-, < hT < 2, and T is a finite-rank operator, then 

lim n s h(I - hQ) ns Aw = lim n s hA(I - hT) n6 w = 0. (52) 
5^0 5^0 



From (I51j) and (|52j) one gets 



< lim(C - l)5hn s < lim \\n s h(I - hQ) n6 Aw \\ = 0. 
<5^0 5^0 



lim 5n$h = (53) 
6^0 



Thus, 

Now (|43p follows from (|50p . (|53p and Theorem [3J Theorem [5] is proved. □ 

12 



3 Numerical experiments 



3.1 Computing us(ts) 

In [3] an DSM Q was investigated with P = A* and the SVD of A was assumed known. 
In general, it is computationally expensive to get the SVD of large scale matrices. In 
this paper, we have derived an iterative scheme for solving ill-conditioned linear algebraic 
systems Au = fg without using SVD of A. 

Choose P = {A* A + a)~ l A* where a is a fixed positive constant. This choice of P 
satisfies all the conditions in Theorem [3l In particular, Q = AP = A(A*A + aI)~ 1 A* = 
AA*(AA* + aiy 1 > is a selfadjoint operator, and T = PA = (A* A + aI)~ 1 A*A > is 
a selfadjoint operator. Since 

\\A*A\\ A 



T 



A + a 



dE 



A 

sup < 1, 

0<\<\\A*A\\ A + a 



where E\ is the resolution of the identity of A* A, the condition h\\T\\ < 2 in Theorem [5] 
is satisfied for all < h < 1. Set h = 1 and P = (A* A + a)" 1 A* in (HQ). Then one gets 
the following iterative scheme: 

u n+l = u n - {A* A + aI)- 1 {A*Au n - A*f s ), u = 0. (54) 

For simplicity we have chosen u$ = 0. However, one may choose uq = vq if vq is known to be 
a better approximation to y than and vq G M^. In iterations (|54p we use a stopping rule 
of discrepancy type. Indeed, we will stop iterations if u n satisfies the following condition 

\\Au n - f s \\ < 1.015. (55) 

The choice of a affects both the accuracy and the computation time of the method. 
If a is too large, one needs more iterations to approach the desired accuracy, so the 
computation time will be large. If a is too small then the results become less accurate 
because for too small a the inversion of the operator A*A + aI is an ill-posed problem since 
the operator A* A is not boundedly invertible. Using the idea of the choice of the initial 
guess of regularization parameter in [2], we choose a to satisfy the following condition: 

5 < <p{a) := \\A(A*A + a^A'fs - fs\\ < 25. (56) 
This can be done by using the following strategy: 

1. Choose a := tii^L as an initial guess for a. 
all/all ° 



2. Compute 4>(a). If a satisfying ([56]) we are done. Otherwise, we go to step 3. 

a 

2(c-l) 



3. If c = > 3 we replace a by 2 (c-i) anc ^ &° back to step 2. If 2 < c < 3 then we 



replace a by 2 ( e -i) anc ^ §>° b&ck to step 2. Otherwise, we go to step 4. 

If c = < 1 we replace a by 3a. If the inequality c < 1 has occured in some 
iteration before, we stop the iteration and use 3a as our choice for a in iterations 
Otherwise we go back to step 2. 
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In our experiments, we denote by DSM the iterative scheme (|54f) . by VRj a Variational 
Regularization method (VR) with a as the regularization parameter and by VR n the 
VR in which Newton's method is used for finding the regularization parameter using a 
discrepancy principle. We compare these methods in terms of relative error and number 
of iterations, denoted by njj er . 

All the experiments were carried in double arithmetics precision environment using 
MATLAB. 

3.2 A linear algebraic system related to an inverse problem for the heat 
equation 

In this section, we apply the DSM and the VR to solve a linear algebraic system used in 
[2]. This linear algebraic system is a part of numerical solutions to an inverse problem 
for the heat equation. This problem is reduced to a Volterra integral equation of the first 
kind with [0, 1] as the integration interval. The kernel is K(s, t) = k(s — t) with 

£-3/2 1 

k (t) = t, — ^exp(--3-). 

Here, we use the value n = 1. In this test in [2] the integral equation was discretized 
by means of simple collocation and the midpoint rule with n points. The unique exact 
solution u n is constructed, and then the right-hand side b n is produced as b n = A n u n 
(see [2]). In our test, we use n = 10,20, 100 and b rit $ = b n + e n , where e n is a vector 
containing random entries, normally distributed with mean 0, variance 1, and scaled so 
that ||e n || = 5rez||&n||- This linear system is ill-posed: the condition number of ^4ioo 
obtained by using the function cond provided in MATLAB is 1.3717 x 10 37 . This number 
shows that the corresponding linear algebraic system is severely ill-conditioned. 



Table 1: Numerical results for the inverse heat equation with 5 re i = 0.05, n = lOi, i = 1, 10. 







DSM 




VR, 




VR„ 


n 


niter 


ll««-»l|a 


niter 


\\us-yh 


n;tei 


\\ u s-y\ 2 


Wvh 


\\y\h 


II y II 2 


10 


3 


0.1971 


1 


0.2627 


5 


0.2117 


20 


4 


0.3359 


1 


0.4589 


5 


0.3551 


30 


4 


0.3729 


1 


0.4969 


5 


0.3843 


40 


4 


0.3856 


1 


0.5071 


5 


0.3864 


50 


5 


0.3158 


1 


0.4789 


6 


0.3141 


60 


6 


0.2892 


1 


0.4909 


6 


0.3060 


70 


7 


0.2262 


1 


0.4792 


8 


0.2156 


80 


6 


0.2623 


1 


0.4809 


7 


0.2600 


90 


5 


0.2856 


1 


0.4816 


7 


0.2715 


100 


7 


0.2358 


1 


0.4826 


7 


0.3405 



Table [T] shows that the results obtained by the DSM are comparable to those by the 
VR„ in terms of accuracy. The time of computation of the DSM is comparable to that of 
the VR n . In some situations, the results by VR n and the DSM are the same although the 
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VR n uses 3 more iterations than does the DSM. The conclusion from this Table is that 
DSM competes favorably with the VR n in both accuracy and time of computation. 

Figure [T] plots numerical solutions to the inverse heat equation for 5 re i = 0.05 and 
$rei = 0.01 when n = 100. From the figure we can see that the numerical solutions 
obtained by the DSM are about the same those by the VR n . In these examples, the time 
of computation of the DSM is about the same as that of the VR n . 




20 40 60 80 100 20 40 60 80 100 



Figure 1: Plots of solutions obtained by DSM, VR for the inverse heat equation when 
n = 100, 5 re i = 0.05 (left) and 5 re i = 0.01 (right). 

The conclusion is that the DSM competes favorably with the VR n in this experiment. 

4 Concluding remark 

Iterative scheme (|54p can be considered as a modification the Landweber iterations. The 
difference between the two methods is the multiplication by P = (A*A + aI)~ 1 . Our itera- 
tive method is much faster than the conventional Landweber iterations. Iterative method 
(|54p is an analog of the Gauss-Newton method. It can be considered as a regularized 
Gauss-Newton method for solving ill-condition linear algebraic systems. The advantage of 
using ([54"}) instead of using (4.1.3) in [2] is that one only has to compute the lower upper 
(LU) decomposition of A* A + al once while the algorithm in [2] requires computing LU 
at every step. Note that computing the LU is the main cost for solving a linear system. 
Numerical experiments show that the new method competes favorably with the VR in our 
experiments. 
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